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Kt  H eXlTT-I  care  iN  alberta 


what  is  primary  health  care  (PHQ? 

Primary  health  care  is  the  first  point  of  COntaCt  of 
individuals  with  the  health  system  - that  is,  where  health  services  are 
mobilized  and  coordinated  to  promote  health,  prevent  illness,  care 
for  common  illness  and  manage  ongoing  problems  (National  Forum 
on  Health,  1997).  Primary  health  care  services  include: 

health  promotion,  illness  and  injury  prevention,  screening,  health 
information,  examinations,  treatment  in  physicians’  offices, 
vaccinations,  home  visits,  nutritional  counselling,  drug  dispensing, 
home  care  and  so  on.  It  involves  a variety  of  health 
professionals  including  family  physicians,  nurses,  public  health 
nurses,  nurse  practitioners,  pharmacists,  mental  health  therapists, 
rehabilitation  therapists,  optometrists,  home  care  providers,  social 
workers  and  others. 
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PREFACE 

The  Umbrella  Alberta  Primary  Health  Care  Project  finds  its  roots  in  the  final 
report  of  the  National  Forum  on  Health  (1997).  The  report  recommended 
reform  in  key  areas,  including  primary  health  care.  It  further  recommended 
creation  of  a health  transition  fund  to  support  evidence-based  pilot  and 
evaluation  projects  in  health  care.  In  response  to  those  recommendations, 
the  federal  government  created  the  Health  Transition  Fund  (HTF)  in  1997. 

It  focused  on  four  key  areas: 

• primary  health  care  reform; 

• integrated  delivery  of  services; 

• home  care;  and 

• pharmacare. 

Following  consultation  with  stakeholders  in  1997,  Alberta  Health  and 
Wellness  chose  to  focus  on  primary  health  care  and  submitted  a proposal 
to  the  federal  government.  Alberta  received  an  $1 1.2  million  HTF  grant  in 
1998  from  the  federal  government,  and  the  Umbrella  Alberta  Primary  Health 
Care  Project  was  created.  An  external  selection  committee  reviewed  primary 
health  care  project  proposals  from  across  the  province,  and  27  were  ultimately 
recommended  for  funding.  Most  of  the  projects  began  in  September  1998 
and  were  completed  by  May  2000.  Extensive  dissemination  activities  by  the 
Umbrella  Alberta  Primary  Health  Care  Project  and  its  27  funded  projects 
commenced  in  August  2000,  including  a provincial  showcase  conference 
and  the  development  of  diverse  products  to  disseminate  the  learning. 

Detailed  information  on  each  of  the  27  projects  can  be  found  at: 
www.health.gov.ab. ca/key/phc/index.htm 

SHARING  THE  LEARNING 

While  the  projects  were  diverse,  a number  of  common  themes  emerged  from 
the  hnal  project  reports  and  evaluation  reports.  Key  findings  in  the  areas  of 
primary  health  care  delivery,  integration,  multi-disciplinary  teams,  access  and 
quality  were  compiled  in  a series  of  booklets,  as  well  as  other  publications, 
designed  to  share  the  findings  provincially  and  nationally  as  a contribution 
to  the  advancement  of  primary  health  care. 

Understanding  and  measuring  quality  in  primary  health  care  is  based  on 
identifying  appropriate  indicators.  Determining  what  these  indicators  are 
rests  on  the  definition  of  primary  health  care.  Primary  health  care  is  the  first 
point  of  contact  and  usually  the  first  level  of  care  that  people  have  with  the 
health  system.  The  timeline  for  these  27  projects  was  a relatively  short  two 
years  and  this  created  significant  challenges  to  assessing  quality  in  a definitive 
way.  However,  this  stand-alone  booklet  shares  important  learning  that  the  27 
projects  can  contribute  to  an  understanding  of  quality  in  primary  health  care 
and  the  issues  related  to  it. 

The  other  materials  can  be  obtained  through  Communications  Branch, 
Alberta  Health  and  Wellness,  22  Floor,  Telus  Plaza  North  Tower,  10025 
Jasper  Avenue,  Edmonton,  Alberta  T5J  2N3.  Telephone  780-427-7164. 

Alberta  Health  and  Wellness  extends  its  thanks  to  the  project  teams,  regional 
health  authorities,  partners,  the  Evaluation  Management  Team  (Howard 
Research  Inc.)  and  the  independent  evaluation  teams  for  their  work  in 
advancing  primary  health  care  in  Alberta.  It  also  thanks  R.A.  Malatest  & 
Associates  Ltd.  and  Dr.  Leslie  Gardner,  who  contributed  to  the  development 
of  this  booklet,  and  our  designers.  Vision  Design  Communications  Inc. 


The  Umbrella  Alberta  Primary 
Health  Care  Project  was 
created  to  further  advance 
primary  health  care  by 
informing  the  practice 
of  primary  health  care  in 
Alberta  and  future  policy 
development. 


PRIMARY  HEALTH  CARE  IN  ALBERTA 


Understanding  and  measuring  quality  in  primary 
health  care  is  based  on  identifying  appropriate 
indicators.  Determining  what  these  indicators  are  rests 
on  the  definition  of  primary  health  care.  Primary  health 
care  is  the  first  point  of  contact  and  usually  the 
first  level  of  care  that  people  have  with  the  health  system. 


In  recent  years,  primary  health  care  has  increasingly 
involved  a range  of  health  providers  and  a focus 
on  a multi-disciplinary  approach  to  care.  Primary  health 
care  is  based  on  a holistic  definition  of  health 
that  recognizes  the  influence  of  social,  economic 
and  environmental  factors  on  a person  S well-being. 


One  of  the  difficulties  in  discussing  quality  as  a separate  concept 
in  terms  of  primary  health  care  is  that  quality  is  inherent  in  many 
of  the  other  dimensions,  such  as  access  and  integration.  During  the  course 
of  the  27  Alberta  primary  health  care  projects,  many  project  personnel 
found  it  more  relevant  to  measure  quality  through  the  assessment 
of  the  other  dimensions,  rather  than  trying  to  measure  quality  directly. 

“Of  all  the  national  dimensions,  quality  may  have  been  the  most  difficult  to  consider 
separately  from  the  other  dimensions  of  access,  integration,  impact  on  populations 
and  cost-effectiveness..  A wide  range  of  approaches  make  assessing  the  quality 
of  primary  health  care  programs  a substantial  challenge." 

2000  Howard  Research  and  Instructional  Systems  Inc.,  Primary  Health  Care: 

Six  Dimensions  of  Inquiry 

The  key  elements  of  primary  health  care  identified  by  the  27  primary  health 
care  projects  which  can  be  used  to  measure  quality  of  care  were: 

• community  involvement; 

• accessibility; 

• availability; 

• comprehensiveness; 

• integration  and  coordination  of  services; 

• accountability; 

• affordability;  and 

• consistent  evaluation. 

Improved  access  to  health  services  was  considered  by  several  projects 
as  a measure  of  quality.  It  was  assessed  from  the  patient/client  perspective 
and  also  as  an  outcome  measure.  For  example,  the  8th  & 8th  Health  Centre 
in  Calgary  offers  24-hour  access  to  urgent  medical  care,  mental  health 
and  public  health  services  and  home  care  for  patients/clients  living 


Utilization  of  clinic  services  increased 
by  68  per  cent  over  the  study  period. 


“While  client  satisfaction  is  a good 
measure  of  quality  in  primary  health 
care,  it  doesn't  replace  the  need  to  ensure 
clinical  quality.  A full  assessment  of 
quality  has  to  happen  over  the  long  term 
and  include  evaluation  of  outcomes. 

The  timeframe  (for  the  project)  was  too 
short  to  adequately  assess  quality  from 
an  outcome  perspective.  Appropriate 
benchmarks  and  indicators  are  difficult  to 
find.  Our  assessments  of  quality  were 
based  on  the  perceptions  of  clients  and 
providers.  Future  assessments  over  the 
long  term  will  include  outcome 
measures.” 

Marion  Relf,  Site  Director,  Northeast 
Community  Health  Centre 
(This  centre,  in  Edmonton,  provides 
a unique  range  of  coordinated 
primary  health  care  services 
to  the  surrounding  community.) 


and  working  in  Calgary’s  downtown  core.  Utilization  of  clinic  services 
increased  by  68  per  cent  over  the  study  period.  Similarly,  the  East  Central 
Health  Authority  project  found  that  quality  was  tied  to  client/patient 
perceptions  of  access  to  services  at  a local  level. 

Assessing  quality  in  primary  health  care  is  a relatively  young  science. 

It  can  be  approached  from  a variety  of  perspectives  and  measured  through 
the  use  of  various  techniques.  Since  those  techniques  are  based  on  methods 
of  evaluating  quality  of  care  in  other  settings  (e.g.  acute  care,  continuing 
care),  it  has  been  challenging  to  develop  appropriate  indicators  of  quality 
within  the  unique  environment  of  primary  health  care  and  to  design 
appropriate  instruments  to  measure  quality  within  that  unique  environment. 

In  this  booklet,  we  summarize  the  experiences  of  the  Umbrella  Alberta 
Primary  Health  Care  projects  in  evaluating  quality  care  using: 

• patient/client  perception  of  quality; 

• provider  perception  of  quality;  and 

• outcome  perception  of  quality. 


PATIENT/CLIENT  PERCEPTIONS  OF  QUALITY 

“Client  satisfaction  is  a valid  measure  of  quality  care  in  a primary  health  care  setting. 

This  is  an  environment  based  on  relationships  with  clients.  They  have  to  be  satisfied 
and  comfortable  with  their  caregivers  or  they  won’t  access  the  services.” 

Marion  Relf,  Site  Director,  Northeast  Community  Health  Centre 

Patient/client  perceptions  of  quality  covered  many  topics  and  were  assessed 
using  a wide  variety  of  indicators.  These  indicators  were  typically  related  to 
those  aspects  of  the  service  that  the  community  in  each  project  values  and 
that  the  projects  were  trying  to  achieve  for  their  patients/clients.  For  example, 
the  Lakeland  Regional  Health  Authority’s  Health  For  All  Project  (Metis 
Settlement),  which  placed  settlement  nurses  in  four  Metis  communities, 
sought  to  increase  knowledge  in  those  communities  about  health  issues  and 
to  have  community  members  take  responsibility  for  their  own  health. 

In  another  example,  the  objective  in  the  Health  Authority  5 project  was 
to  increase  access  to  primary  health  care  services  in  small  rural  communities. 
The  Community  Outreach  in  Pediatrics/Psychiatry  and  Education  Program 
(COPE)  (Project)  in  Calgary  brought  together  providers,  teachers  and  learning 
specialists  to  best  meet  the  needs  of  children  with  mental  health  issues. 


The  Healthy  Families  Project  in  the  Capital  Health  Authority  (Edmonton) 
recognized  that  the  young  parents  for  whom  they  were  developing  services 
often  did  not  have  sufficient  personal,  family  or  emotional  resources,  nor 
sufficient  knowledge  of  parenting,  to  care  for  a child.  The  project 
determined  that  a comprehensive  and  long-term  home  visitation  program 
was  required.  A patient/client  satisfaction  survey  was  administered  in  focus 
groups  to  assess  the  quality  of  the  services  delivered.  Questions  centred  on 
the  Family  Visitor’s  sensitivity  to  the  patient/client’s  culture  and  beliefs, 
increased  use  of  other  community  programs,  improvements  in  parenting 
and  improvements  in  child  development  gained  as  a result  of  the  program. 
Almost  all  the  families  involved  in  the  program  expressed  a high  level 
of  satisfaction  with  the  services  provided  as  measured  by  these  indicators. 

The  Lakeland  Regional  Health  Authority’s  Integrated,  Community-based 
Palliative  Care  Program  developed  a multi-disciplinary  team  of  care 
professionals  skilled  in  palliative  care  to  provide  consultation  advice  to 
physicians  and  nurses  in  one  community  in  the  region.  Assessing  quality  of 
care  in  the  program  included  in-person  interviews  with  families  of  deceased 
persons  who  had  been  part  of  the  program.  The  questions  in  the  family  survey 
were  open-ended  and  designed  to  determine  how  the  program  had  helped  and 
how  it  could  be  improved.  All  of  the  individuals  interviewed  indicated 
overwhelming  satisfaction  with  the  program. 

“She  wanted  to  be  pain-free  and  at  home.  And  the  Palliative  Care  Program  helped 
us  achieve  that  goal,  it  really  did.  The  care  from  the  Palliative  Care  Program. . . 
you  couldn’t  have  asked  for  anything  better.  It  was  terrific.” 

Family  Caregiver,  Lakeland  Regional  Health  Authority  Integrated 
Community-based  Palliative  Care  Program 


“The  settlement  nurse  is  sincere,  loving  and 
always  aware  of  cultural  needs.  Members 
are  really  satisfied  with  the  nurse.  She  is 
open,  friendly  and  keeps  the  confidentiality 
that  is  so  needed.” 

Buffalo  Lake  Settlement  Council 
Representative,  Lakeland  Regional 
Health  Authority  Health  For  All 
(Metis  Settlement)  Project 


The  Lakeland  Regional  Health  Authority  Health  For  All  (Metis  Settlement) 
Project  also  found  that  the  perceived  level  of  competence  and  trustworthiness 
of  the  settlement  nurses  was  an  indicator  of  quality  from  the  patient/ client’s 
perspective. 

Health  Authority  5 conducted  a patient/client  survey  to  measure  health 
status  and  health  behaviour  before  and  after  the  intervention  in  four 
communities.  One  of  the  questions  in  the  survey  asked  patients/clients 
to  rate  the  current  quality  of  service  received  from  the  health  region. 

The  results  showed  a statistically  significant  increase  in  patient/client 
perception  of  the  quality  of  service  received  in  one  community. 

Patients/ clients  visiting  the  clinics  were  also  asked  a series  of  questions 
to  determine  their  satisfaction  with  services  provided.  The  questions  focused 
on  location,  benefit  to  the  community,  recommending  the  service  to  others 


“He’s  a good  nurse.  He’s  always  there 
when  you  need  him.  He  never  turns  you 
away.  If  you  have  a problem,  he’ll  always 
take  the  extra  time  and  effort  to  deal 
with  it,  with  you.  Keep  him  here, 
he’s  a good  guy.’’ 

Client  of  the  Calgary  Urban  Project 
Society  (CUPS)  Drop-in  Centre 
where  the  nurse  practitioner 
conducts  outreach  programs 


and  whether  they  would  use  the  service  again.  In  response  to  all  questions, 
the  majority  of  patients/clients  said  they  were  “very”  or  “extremely”  satisfied 
with  the  services  provided. 

Patients/clients  in  the  Evaluation  of  a Primary  Health  Care  Clinic  Project 
listed  the  following  as  their  top  priorities  with  delivery  of  service: 

• comprehensive  service; 

• good  rapport  with  physician/health  care  provider; 

• immediate  referrals  to  specialists  if  needed; 

• friendly  care  providers; 

• excellent  follow-up; 

• prompt  service;  and 

• provider  availability. 

Community  members  in  the  same  project  identified  the  following  measures 
of  satisfaction  with  service  delivery: 

• competency  of  the  physician/health  care  provider; 

• hours  of  service; 

• trusting  relationship  with  physician/health  care  provider; 

• proximity  to  home; 

• less  waiting  time;  and 

• continuity  with  family  physician. 

Provider  competence  was  an  important  indicator  of  patient/client 
satisfaction  for  many  of  the  projects.  The  Calgary  Urban  Project  Society’s 
(CUPS)  project  integrated  a nurse  practitioner  into  the  primary  health  care 
model  in  their  community  health  centre.  Patients/clients  were  surveyed 
during  three  time  periods  using  a survey  adapted  from  another  inner-city 
health  program.  The  results  of  the  survey  indicated  that  an  important 
measure  of  quality  for  the  patients/clients  was  the  competence  and 
personality  of  the  nurse  practitioner. 


PROVIDER  PERCEPTIONS  OF  QUALITY 

Providers  assessed  quality  of  care  in  the  projects  against  a variety 
of  indicators  including  the  availability  of  training  and  education,  the  impact 
of  a collaborative,  multi-disciplinary  environment  and  the  presence 
of  practice  guidelines  and  protocols. 

In  the  Lakeland  Integrated,  Community-based  Palliative  Care  Project, 
the  palliative  care  nurse  consultant  and  the  physician  consultant  received 
training  and  education  to  ensure  the  quality  of  service  provided.  A survey 
asked  health  professionals  how  the  consultations  impacted  the  way  care 
was  provided,  and  whether  the  overall  quality  of  palliative  care  to  patients 
was  improved  since  the  pilot  project  began.  The  results  indicated  that  the 
consultations  had  helped  caregiver  satisfaction  increase.  There  was  a feeling 
that  better  quality  care  was  being  provided  through  improved  pain  and 
symptom  management,  narcotic  rotation  and  terminal  sedation. 

“(It)  hugely  improved  my  understanding  and  management  of  care  in  all  areas  - medical, 
pain  and  nausea  control,  social  and  religious.  Yet  they  [the  consultative  team]  left  me 
feeling  in  control.” 

Health  care  provider,  Lakeland  Integrated  Community-based  Palliative 
Care  Project 

Projects  also  found  that  the  quality  of  their  service  increased  with 
the  collaborative  development  of  clinical  practice  guidelines.  For  example, 
the  primary  health  care  team  at  the  CUPS  Health  Centre  in  Calgary  met 
to  develop  clinical  practice  guidelines  for  ten  selected  conditions  as  agreed 
upon  by  the  physicians  and  nurse  practitioner.  A change  in  provincial 
legislation  during  the  project  meant  that  physician  supervision  of  the  nurse 
practitioner  was  not  required  and  monitoring  adherence  to  the  guidelines 
became  inappropriate.  However,  the  guidelines  began  a process  that  would 
become  the  de  facto  basis  for  continuous  quality  improvement 
in  the  program.  They  also  served  as  a tool  for  consultation  by  staff 
and  for  orientation  of  new  staff  New  protocols  and  guidelines  are  being 
developed  continuously. 

The  Lakeland  Integrated  Community-based  Palliative  Care  Program 
developed  clinical  practice  protocols  that  identified  best  practices  for 
palliative  care  for  various  providers.  The  document  provided  a standard 
for  quality  palliative  care  across  the  region.  This  facilitated  expanded 
knowledge  regarding  palliative  care  and  improved  access  to  palliative  care 
service  by  patients/families.  This  included  increased  involvement  and 
communication  among  all  groups  involved  in  providing  palliative  care. 
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“The  guidelines  are  part  of  a living 
document.  New  guidelines  are  constantly 
being  developed  in  response  to  recurrent 
clinical  diagnoses.  They  give  us  a 
benchmark  for  continued  evaluation 
of  clinical  quality.” 

Lorraine  Melchior,  Executive  Director, 
CUPS  Community  Health  Centre 
in  Calgary 


“We  believe  that  the  small  changes  that 
have  taken  place  during  the  pilot  were 
significant  steps  to  improving  end-of-life 
care.  Ongoing  success  will  foster 
acceptance  and  change  in 
current  practice.'' 

Consultation  Team,  Lakeland  Regional 
Health  Authority  Integrated 
Community-based  Palliative 
Care  Program 


Quality  was  also  assessed  in  terms  of  achieving  a multi-disciplinary 
environment.  The  Primary  Health  Care  Collectives  Project  in  Edmonton 
formed  six  teams  of  physicians,  pharmacists  and  home  care  nurse  managers 
to  improve  medication  use  by  high-risk  individuals  in  the  community. 

The  assessment  of  provider  satisfaction  centred  on  their  role  recognition 
and  their  experiences  in  the  collective.  Providers  were  asked  about  their 
expectations  at  the  beginning  of  the  project,  and  then  about  their  actual 
experience  every  two  months  during  the  project.  They  were  also  asked  about 
the  perceived  impact  of  collaboration  on  such  things  as  quality  of  patient  care. 

The  results  indicated  that  the  team  approach  to  care  significantly  exceeded 
providers’  expectations  on  aspects  of  quality  of  care  related  to: 

• clear  professional  roles; 

• feeling  part  of  a team; 

• feeling  that  working  with  others  was  helpful;  and 

• receiving  enough  patient  information  and  improving  patient/client 
health  status. 


Other  domains,  such  as  job  satisfaction,  did  not  exceed  expectations, 
but  the  evaluation  noted  that  this  might  have  been  the  result  of  high 
expectations  going  into  the  project. 


OUTCOME  PERCEPTIONS  OF  QUALITY 

Evaluating  quality  of  care  by  assessing  health  outcomes  was  difficult  given 
the  short-term  nature  of  the  projects.  However,  several  projects  successfully 
built  this  into  their  evaluation  process. 

The  Lakeland  Regional  Health  Authority  Integrated  Community-based 
Palliative  Care  Program  (Project)  conducted  a chart  review  of  acute  care 
and  home  care  patients/clients  before  and  after  the  program.  Information 
collection  forms,  designed  by  an  evaluation  team,  tracked  patient  assessment 
and  the  management  of  several  symptoms  commonly  experienced 
by  patients  suffering  from  terminal  cancer.  The  results  were  measured  against 
symptom  management  guidelines.  For  example,  one  guideline  states  that 
Demerol  should  be  avoided  for  pain  management.  The  audit  showed  that 
before  the  program,  18  per  cent  of  patients/clients  were  receiving  Demerol. 
After  the  program,  no  patients/clients  were  receiving  Demerol. 

The  Primary  Health  Care  Collectives  Project  (PHCC)  used  several 
outcome-focused  measures  to  assess  quality  of  care.  The  PHCC  focused 
on  significantly  improving  medication  use  by  high-risk  individuals,  using 
six  teams  (collectives)  consisting  of  a family  physician,  a pharmacist 


and  a home  care  nurse  manager.  They  measured  quality  through  the  number 
and  nature  of  drug-related  problems  identified  and  resolved,  using 
the  Medication  Appropriateness  Index,  and  through  health  status  measures 
(SF-12  and  Health  Utilities  Index).  The  project  achieved  its  intended  goal 
of  significantly  improving  patient/client  compliance  with  medication 
regimens,  with  no  differences  across  Collectives.  Health  status 
did  not  significantly  improve  but  was  maintained  from  baseline  to  follow-up. 
Given  that  the  health  status  of  the  high-risk  individuals  in  the  project  might 
be  expected  to  decline  over  time,  this  was  a positive  outcome. 

The  project  also  gathered  qualitative  data  that  illustrated  the  impact 
of  the  teams  on  health  status  of  the  patients/clients. 

“A  person  with  HIV  had  a long-standing  mycobacterial  infection  in  his  lymph  glands. 

His  case  conference  noted  that  his  viral  load  had  tripled  since  August.  Through 
the  medication  review  process  it  was  found  that  he  was  ‘picking  and  choosing’  among 
his  medications:  taking  his  pain  meds  but  not  his  TB  meds.  The  pharmacist  reduced 
the  number  of ‘packages’ so  that  theTB  meds  were  packaged  along  with  the  pain 
meds.  She  also  suggested  changing  to  taking  them  twice  daily  instead  of  spread 
out  throughout  the  day.  The  person  started  taking  the  medications  appropriately 
and  the  infection  came  under  control.” 

Independent  Evaluation,  Final  Report,  Primary  Health  Care  Collectives. 

(The  Primary  Health  Care  Collectives  Project  focused  on  improving  medication 
use  by  high-risk  individuals,  through  six  collectives.  Each  collective  consisted 
of  a family  physician,  pharmacist  and  a home  care  nurse  manager.) 
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The  Healthy  Families  Project  in  the  Capital  Health  Authority  used  five 
standardized  instruments  to  measure  the  effects  of  participation  in  the 
program.  The  instruments  were  administered  at  baseline  and  following 
participation  in  the  program.  The  research  questions  that  these  instruments 
were  used  to  answer  were  directly  related  to  the  objectives  of  the  program; 
for  example,  one  objective  of  the  program  was  to  promote  healthy  childhood 
development.  The  Denver  Developmental  Screening  Test  II  was  selected  to 
evaluate  the  impact  of  the  program  in  this  area.  The  results  showed  a positive 
impact  of  the  project  on  the  development  of  children  participating 
in  Healthy  Families. 


CHALLENGES 


Six  challenges  to  measuring  quality  of  care  in  a primary  health  care  setting 
were  identified  by  some  of  the  projects.  These  were: 

The  short-term  nature  of  pilot  projects  limited  the  assessment 
of  health  outcomes 

The  active  life  of  the  Primary  Health  Care  Collectives  was  just  nine  months. 
Given  the  short  time  line,  the  standardized  assessment  tool  of  health  status 
and  health  service  utilization  approached  statistical  significance, 
but  did  not  show  significance.  The  project  staff  believe  that  had  the  project 
run  long  enough  to  recruit  the  full  300  client  participants  originally 
anticipated,  they  would  have  been  able  to  show  significance  using  the  tools. 

Finding  valid  standardized  instruments  that  were  appropriate 
to  primary  health  care 

Initial  pilot  testing  of  patient/ client  satisfaction  instruments  at  the  Northeast 
Community  Health  Centre  (NECHC)  in  Edmonton,  resulted  in  a search 
for  a more  appropriate  instrument.  A “Services  Satisfaction  Survey”  was 
recommended  and  is  being  modified  and  piloted  to  make  it  more 
appropriate  for  the  population  in  Northeast  Edmonton.  Focus  groups  were 
conducted  with  patients/ clients  and  found  they  were  generally  satisfied  with 
the  quality  of  services  at  the  NECHC.  A number  of  areas  for  improvement 
were  also  identified. 

The  literacy  level  of  specific  target  groups 

A major  challenge  to  developing  evaluation  instruments  in  the  Calgary 
Urban  Project  Society  (CUPS)  project  was  the  literacy  level  of  the  clientele. 
The  research  assistant  was  often  asked  to  verbally  administer  the  survey. 
Further,  once  patients/clients  had  completed  the  initial  survey,  they  were 
unwilling  to  complete  it  again  on  subsequent  visits  because  they  felt 
it  was  too  long  or  difficult  to  complete.  Many  patients/clients  were  also 
skeptical  of  questioning  and  were  unwilling  to  provide  any  personal 
information  including  their  names,  or  they  would  give  different  names 
each  time  they  came  into  the  Centre.  The  transient  nature  of  many 
of  the  people  living  in  the  area  being  served  by  the  program  prohibited 
a pre-post  evaluation  design. 


Establishing  data  collection  on  quality  as  part  of  the  service  delivery 

The  family  visitors  in  the  Healthy  Families  Project  in  Capital  Health  were 
required  to  collect  outcome  data,  something  not  expected  at  the  beginning 
of  the  program.  The  program  was  initially  concerned  about  the  potential 
misfit  between  the  research  methodology  of  data  collection  and  the 
service-delivery  relationship-based  philosophy  of  the  program.  Further, 
the  large  number  of  instruments  and  the  late  start  to  collecting  data  made 
the  task  of  catch-up  somewhat  onerous.  However,  once  data  collection  was 
established  on  a regular  basis,  evaluation  became  the  norm  as  part  of  family 
visits.  And  the  family  visitors  welcomed  the  rapid  feedback  on  individual 
family  results. 

Establishing  the  trust  necessary  for  data  collection 

The  CUPS  survey  was  administered  by  the  regular  receptionist  at  the  Centre 
who  acted  as  the  research  assistant.  This  was  important  as  there  was  a great 
deal  of  suspicion  among  the  Centre’s  inner-city  clientele  relating  to  mental 
health  issues  or  past  experiences  with  the  law.  The  high  level  of  trust 
between  the  patients/clients  and  the  receptionist  increased  the  willingness 
of  patients/clients  to  participate  in  the  survey  process. 

Identifying  all  costs  and  consequences  associated  with 
cost-effectiveness 

It  was  often  difficult  for  projects  to  track  all  costs  associated  with  the  project, 
particularly  if  more  than  one  organization  or  agency  was  involved. 

As  a result,  cost-effective  analysis  was  limited  and  often  relied  on  proxies 
or  anecdotal  evidence.  For  instance,  the  Primary  Health  Care  Collectives 
Project  has  some  evidence  that  the  collectives  were  able  to  decrease  usage 
of  emergency  services  and  decrease  visits  to  the  physicians’  offices.  Similarly, 
the  evaluation  of  the  8th  & 8th  Health  Centre  in  Calgary  suggested  that 
hospitalization  may  have  been  averted  for  a significant  number  of  mental 
health  clients  accessing  mental  health  services  through  the  centre,  which 
intuitively  indicates  overall  savings  to  the  health  system. 


Northeast  Community  Health  Centre, 
Edmonton 
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[conclusions] 


“Without  a sense  of  clarity  about  what 
we  mean  and  indicators  that  can  be 
tracked  and  measured,  we  have  no  way 
to  determine  the  quality  of  our  services 
and  to  whom  we  think  we  should  be 
providing  them.Without  all  that,  we  end 
up  paying  lip  service  to  it.” 

Jeanne  Besner,  Director  of  Primary 
Care  Development,  Calgary  Regional 
Health  Authority 


Despite  the  challenges,  the  experiences  of  the  27  pilot  and  evaluation 
projects  in  Alberta  have  contributed  significantly  to  developing  a body 
of  knowledge  on  evaluating  quality  in  primary  health  care.  A broad  range 
of  approaches  and  instruments  were  developed,  tried  and  revised  in  an  effort 
to  ensure  that  high  quality  services  were  delivered. 

Information  on  the  projects  that  used  these  tools  is  available  on  the  Alberta 
Health  and  Wellness  website. 

As  noted  by  a family  caregiver  in  the  Lakeland  Regional  Health  Authority 
Integrated  Community-based  Palliative  Care  Program,  “It’s  comforting 
to  know  that  there  are  people  out  there  who  strive  to  make  it  better...” 


Funding  and  support  to  produce  this  booklet  has  been  provided  by  Health  Canada 
through  the  Health  Transition  Fund  and  by  Alberta  Health  and  Wellness. The  views 
expressed  herein  do  not  necessarily  represent  the  official  policies  of  Health  Canada 
or  the  Government  of  Alberta. 
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